Data centers traditionally use a hierarchical organization of computing units to handle computing tasks. In this organization, the data center may include a plurality of racks. Each rack includes a plurality of computing units (such as a plurality of servers for implementing a network-accessible service). Each rack may also include a rack-level switching mechanism for routing data to and from computing units within the rack. One or more higher-level switching mechanisms may couple the racks together. Hence, communication between computing units in a data center may involve sending data “up” and “down” through a hierarchical switching structure. Data centers physically implement these communication paths using hardwired links.
The hierarchical organization of computing units has proven effective for many data center applications. However, it is not without its shortcomings. Among other potential problems, the hierarchical nature of the switching structure can lead to bottlenecks in data flow for certain applications, particularly those applications that involve communication between computing units in different racks.